From actions to habits: neuroadaptations leading to dependence.

Recent work on the role of overlapping cerebral networks in action selection and habit formation has important implications for alcohol addiction research. As reviewed below, (1) these networks, which all involve a group of deep-brain structures called the basal ganglia, are associated with distinct behavioral control processes, such as reward-guided Pavlovian conditional responses, goal-directed instrumental actions, and stimulus-driven habits; (2) different stages of action learning are associated with different networks, which have the ability to change (i.e., plasticity); and (3) exposure to alcohol and other addictive drugs can have profound effects on these networks by influencing the mechanisms underlying neural plasticity.

A ddiction is a series of misguided actions. Yet how the brain selects and generates actions has received surprisingly little attention in addiction research. In recent years, considerable progress has been made in identifying the neural circuits responsible for the control of goaldirected actions and habit formation. It is becoming increasingly clear that drugs of abuse can alter these neural pathways. This article discusses the mechanisms underlying rewardguided action selection and their implications for research on alcohol addiction.

The Organization of CorticoBasal Ganglia Networks
Understanding how the brain generates actions must begin with a discussion of the corticobasal ganglia networks. 1 These networks form a hierarchy for motivated behavior (Swanson 2000;Knowlton 2005, 2006), which consists of variations on a basic motif, a prototypical network critical for behavioral selection. In this network, glutamatergic (excitatory) projection neurons from the cerebral cortex, a highly layered structure, send axons to the nuclei underneath, commonly known as the basal ganglia, which contain γaminobutyric acid (GABA) ergic (inhibitory) projection neurons. The inhibitory outputs from the basal ganglia, in turn, are directed at down stream structures in the brainstem and in various thalamic nuclei whose pro jections reenter the cortex.
There is reason to believe that the basal ganglia circuits and their intrinsi cally generated oscillations are respon sible for the generation and selection of behavioral programs; and the varia tions in patterns of connectivity and in the expression of key proteins like membrane receptors may be tailored for different types of global control processes, as described below (Gerdeman et al. 2003;. A striking feature of such control pro cesses is that they can be measured behaviorally using specific tests. As recent research has shown, normal mechanisms of learning and memory are usurped by exposure to addictive drugs, so that instead of serving normal biological needs they defect to the purpose of drug seeking (Hyman et al. 2006). There is no consensus, however, on precisely what type of learning process is usurped by addictive substances. Current hypotheses focus on the enhancement of craving, or incentive sensitization (Robinson and Berridge 2003), and on the avoidance of harmful consequences of with drawal, or allostasis (Le Moal and Koob 2007). These hypotheses largely neglect the central issue of how actions are selected. One reason for this neglect is that the chief behavioral measures in the field (e.g., selfadministration and conditioned place preference 2 ) 1 This and other technical terms can be found in the Glossary, Conditioned place preference is a commonly used tech nique to evaluate preferences for environmental stimuli that have been associated with a reward. In general, this proce dure involves several trials where the animal is presented with the reward (e.g., food or the effects of a drug of abuse) paired with placement in a distinct environment containing various cues (e.g., tactile, visual, and olfactory). When later tested in the normal state, approaches and the amount of time spent in the compartments previously associated with reward serve as an indicator of preference and a measure of reward learning. lack sufficient analytical power to isolate contributions of distinct neural networks. As discussed below, a major challenge in addiction research is to understand the mechanisms underly ing these behavioral control processes and how they are affected by exposure to alcohol and other drugs.

Three Modes of Behavioral Control
What, then, are these control processes and why are they so important for understanding alcohol addiction? In the study of behavior guided by rewards (i.e., appetitive behavior), researchers are now able to distinguish three major modes of behavioral control with simple experimental tests. These three modes are Pavlovian approach, 3 goaldirected action, and habit. Although these are rather broad classes of behavioral control with simple operational definitions, they shed considerable light on the integrative functions of the corticobasal ganglia networks.
Preparatory appetitive Pavlovian behaviors (e.g., approaching location of reward and stimuli that predict reward) and goaldirected instrumen tal actions are both controlled by the anticipation of the reward. For both, reducing the value of the reward (e.g., by selective satiety, in which the ani mal is sated on the particular reward offered but not other rewards) or taste aversion induction (in which a particular food is paired with an injection of lithium chloride that results in gastric discomfort) can reduce per formance (Colwill and Rescorla 1985;Yin and Knowlton 2002). In both, too, performance is controlled by a predictor of reward and the reward itself. But for Pavlovian approach, the predictor of reward is a stimulus arranged by the experimenter and entirely independent of the animal's behavior, whereas in instrumental behavior the predictor is the selfgen erated action by the animal. This dis tinction is revealed by direct manipu lation of the postulated contingencies (e.g., increasing the probability of reward independent of the predictor, be it a particular action in the case of instrumental learning or a stimulus in the case of Pavlovian conditioning) (Hammond 1980;Schwartz and Gamzu 1977). Manipulating the rela tionship between stimulus and out come specifically affects Pavlovian behavior, whereas manipulating the action-outcome relationship specifi cally affects instrumental behavior (Dickinson 1994(Dickinson , 1997Schwartz and Gamzu 1977).
Habit, a third mode of behavioral control, is not affected by changes in outcome value. Habits persist even if the reward becomes less attractive or if the action is not necessary to earn the reward. Unlike appetitive Pavlovian conditional responses, which are controlled by the stimulus-outcome contingency, all instrumental behaviors initially are goal directed and controlled by the action-outcome contingency. The performance of such actions is exquisitely sensitive not only to its causal efficacy (i.e., by the extent to which the outcome depends on the action) but also to the value of the ensuing consequence (Dickinson 1985;Dickinson and Balleine 1993;Knowlton 2005, 2006). Under certain conditions, such as extensive training, however, such goaldirected actions are transformed into habits.
As shown by a number of studies in the last two decades, habitual con trol of instrumental behavior emerges gradually with repeated performance and is relatively unaffected by changes either in outcome value (e.g., devalu ation) or in instrumental contingency (Adams 1982;Adams and Dickinson 1981). Thus, once lever pressing for a sucrose reward becomes habitual in this sense, induced taste aversion or unlimited exposure to sucrose prior to a probe test--conducted with the lever extended but without the pre sentation of a reward--will not reduce the rate of lever pressing compared with controls that did not receive the devaluation treatment.
This basic distinction is supported by a series of studies from Yin and colleagues (2004Yin and colleagues ( , 2005aYin and colleagues ( ,b, 2006, who established a functional dissocia tion between associative and sensori motor striata in the control of instru mental actions. They showed that the associative or medial striatum (similar to most of the caudate nucleus in primates) is critical for the early, goal directed stage of action learning, whereas the sensorimotor or lateral striatum (similar to the putamen in primates) is more critical for the later, more habitual stage (see figure 1). Together with studies of other struc tures in these networks Corbit et al. 2001Corbit et al. , 2002, this line of research has established that control over instru mental behavior lies with the associa tive corticobasal ganglia network in the early stages of learning but switches to the sensorimotor cortico basal ganglia network in later stages Knowlton 2005, 2006;Wickens et al. 2007a,b).
With respect to the neural adapta tions that lead to alcohol depen dence, then, the key question is, Which control processes are affected by alcohol as casual drinking becomes compulsive drinking? Drugs of abuse can enhance Pavlovian approach behavior (e.g., approaching environ mental stimuli associated with reward), which is largely mediated by the ven tral striatum (nucleus accumbens) and the associated corticobasal ganglia circuit (Corbit et al. 2001;Day et al. 2007;Hyman et al. 2006;Parkinson et al. 2000). In fact, because of the inability to isolate Pavlovian from instrumental modes of behavioral control, current research on addiction has focused almost exclusively on the nucleus accumbens; but we now know that this is only part of the story. As reviewed above, the corticobasal ganglia networks, which involve the medial (associative) and lateral (sen sorimotor) striatal regions above the nucleus accumbens, are responsible for instrumental control processes (see figure 2). Thus, previous work has, by and large, neglected the con tributions of the associative and sen sorimotor networks in the study of addiction.

Implications for Alcohol Addiction
A trademark of habitual behavior is that the expected value of the outcome does not affect the behavior. It is as if the value of the outcome has become fixed, so that even if alcohol consump tion is associated repeatedly with aver sive consequences, such consequences do not alter the performance of the action itself. For this reason, habits have been viewed by some researchers as an inter mediate stage before the development of compulsivity (Everitt and Robbins 2005). In the case of alcohol consump tion, such a model would emphasize first a shift from casual drinking to habitual drinking, followed by a shift to compulsive drinking. Nonetheless, although the process of habit formation bears a certain resemblance to addiction, addictive behaviors are not the same as enhanced habits (Yin and Knowlton 2005). At first glance, both develop after repeated exposures, and both are insensitive to outcome devaluation. But there are important differences as well. For example, habitual behavior is easily extinguished when the reward is no longer delivered, whereas compulsive behavior is very resistant to extinction (Mowrer 1960). Thus, whereas decades of work has identified the distinct con trol processes outlined above, we still have little understanding of how these processes interact in producing normal behavior, which rarely is dominated by one process alone. Compulsive behavior, for example, is probably an amalgamation of Pavlovian and instrumental processes.
Appetitive Pavlovian instrumental interactions can take a number of forms. In all, stimuli with incentive value increase the likelihood of action for reward. Although conditioned reinforcement sometimes refers to actioncontingent stimuli, Pavlovian instrumental transfer always measures the effect of actionindependent stimuli. In conditioned reinforcement, cues produced by instrumental actions can form associations with the reward; and after repeated pairing they become viable reinforcers for the actions (Mowrer 1960). For compulsive drinking, con ditioned reinforcement (the feel of the bottle, the taste of alcohol) can play an important role. In Pavlovian instrumental transfer, cues that inde pendently predict reward can elicit cen tral motivational states that enhance instrumental performance. For example, the environmental stimuli associated with drinking (e.g., the sight of a bar) can trigger craving for alcohol and, in turn, alcoholseeking behavior. Much of the power of advertising, for exam ple, probably derives from the ability of Pavlovian stimuli to trigger motiva tional states that enhance the selec tion of certain actions. The nucleus accumbens is known to play a critical role in Pavlovian instrumental transfer; lesions of this area selectively abolish transfer (Corbit et al. 2001). Interestingly, recent work (Corbit and Janak 2007) has also implicated the dorsal striatum. The sensorimotor striatum in particular appears to play a critical role in the ability of rewardpredicting cues to enhance instrumental lever pressing. Such results suggest the possibility of interactions between ventral and more dorsal striatal regions in Pavlovian instrumental interactions.

The Role of Plasticity
It is possible that all addictive drugs, including alcohol, can affect the capacity for change (i.e., plasticity) in the cor ticobasal ganglia networks, thereby altering normal learning processes that are critical for selecting and controlling actions. Although plasticity at all parts of the corticobasal ganglia network may be involved in addiction, the striatum appears to be the critical node where massive excitatory inputs are trans formed into an inhibitory output that ultimately controls behavior (Lo and Wang 2006;Nauta 1989). The gluta matergic transmission can be altered, both presynaptically, in the amount of glutamate released from the axon terminal, and postsynaptically, in the trafficking and expression of various glutamate receptors.
Recent studies (Jedynak et al. 2007;Nelson and Killcross 2006;Porrino et al. 2004) show that exposure to drugs like cocaine and amphetamine can result in significant plasticity in the striatum and potentially accelerate the initial shift from actions to habits. Alcohol may produce similar effects. Acute application of alcohol to brain slices can reverse the direc tion of plasticity in the associative striatum (Yin et al. 2007). Thus, a train of stimulation that normally leads to increased activity in a striatal region critical for goaldirected actions results in longterm depres sion instead. One interpretation of these results suggests that the reversal of striatal plasticity could promote habit formation by reducing the over all synaptic strength of the associative striatum, which is a critical compo nent of the brain's system for the con trol of goaldirected actions. Previous work Yin et al. 2004Yin et al. , 2005aYin et al. ,b, 2006 showed that disrupt ing the network for goaldirected actions results in a switch to a habitu al mode of behavioral control, and vice versa. It remains to be seen if alcohol is able to promote habit for mation in vivo by targeting this mechanism.

Conclusions
The preliminary conceptual framework and the behavioral tests discussed here suggest a number of promising avenues for future study. Researchers can mea sure, for example, the effects of alcohol on each of these control processes, on their interactions, and on the underly ing neural substrates at the cellular level as well as at the level of neural circuits. Further work also can investigate the effects of particular factors (e.g., stress) on susceptibility to addiction and to  Figure 2 The corticobasal ganglia networks. An illustration of the major corticostriatal projections and dopaminergic projections in terms of the four major cortico basal ganglia networks and their corresponding behavioral functions. Emphasis is placed on the spiraling midbrain-striatum-midbrain projections, which allows information to be propagated forward in a hierarchical manner. Note that this is only one possible neural implementation; interactions via different thalamo-cortico-thalamic projections also are possible (Haber 2003). relapse using the same strategy. The extent of our ignorance in these areas is considerable. An exciting and chal lenging path lies ahead. ■

Financial Disclosure
The author declares that he has no competing financial interests.